Color image sensor array with color crosstalk test patterns

ABSTRACT

An integrated circuit comprises a semiconductor substrate and a color image sensor array on the substrate. The color image sensor array has a first configuration of color pixels for collecting color image data, and at least one crosstalk test pattern on the substrate proximate the color image sensor array. The crosstalk test pattern includes a plurality of color sensing pixels arranged for making color crosstalk measurements. The test pattern configuration is different from the first configuration.

This application is a continuation of U.S. patent application Ser. No.12/909,154, filed Oct. 21, 2010, which is expressly incorporated hereinby reference in its entirety.

FIELD

This disclosure relates generally to semiconductor devices andfabrication methods, and more specifically to color image sensors andtheir fabrication.

BACKGROUND

Color image sensors are used in a variety of electronic devices whichcapture color image data and convert the optical image to electricalsignals. For example, color image sensors are commonly used in digitalcameras, including standalone still image and video cameras, as well asmulti-function devices with camera functions, such as cellular phones,smart phones, personal digital assistants, web cams, laptop computers,and the like. The term “camera”, as used herein refers to any suchelectronic imaging device.

A common form of color image sensor has an array of picture elements(pixels), each covered by a respective color filter. For example, thecolor filters may be arranged in a Bayer pattern, having alternatingred-green and green-blue rows. The red filters pass red light andideally block blue and green light. The green filters pass green lightand ideally block blue and red light. The blue filters pass blue lightand ideally block red and green light.

However, a variety of factors may result in crosstalk betweenneighboring colored pixels and between colors. For example, spectralcrosstalk may result from imperfect blocking of green or blue light by ared filter, imperfect blocking of red and blue light by a green filter,or imperfect blocking of red and green light by a blue filter. Opticalcrosstalk occurs when a given pixel receives light that should ideallyonly reach a nearby pixel. This may occur, for example, if the incominglight has a non-perpendicular light incidence angle. Other sources ofoptical cross talk include light scattering and reflection at colorfilter array (CFA) and micro-lens boundaries, interconnect metal wires,and light diffraction due to the small size of each color filter pitch.Additionally, there is electrical cross talk, caused by photo-carrierdiffusion in the silicon substrate beneath pixels of different colors.

Color crosstalk results in degradation of sensor spatial resolution.Color information on one pixel is contaminated by information of itsneighbors. This results in degradation of color fidelity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B show two examples of integrated circuits (IC's) havingcolor crosstalk test patterns proximal to the primary color image sensorarray.

FIG. 2 is a block diagram of apparatus for generating color crosstalkcorrection coefficients including an IC as shown in FIG. 1A or 1B.

FIG. 3 is a block diagram of a camera including the IC of FIG. 2.

FIG. 3A is a block diagram of a camera with a variation of the sensor ICof FIG. 3.

FIG. 3B is a block diagram of a camera having the sensor IC of FIG. 3Aand an additional processor.

FIG. 4 is a variation of the apparatus of FIG. 2, having coefficientstorage in the IC.

FIG. 5 is a block diagram of a camera including the IC of FIG. 4.

FIG. 6 is a block diagram of an image processing system including colorcrosstalk correction.

FIG. 7 is a diagram of a monochrome test pattern for collecting testdata without color crosstalk.

FIG. 8 is a diagram of a test pattern having a single pixel of a firstcolor surrounded by a second color.

FIGS. 9 to 12 are diagrams of test patterns having two colors, withvertical, horizontal, and diagonal interfaces.

FIGS. 13 to 16 are diagrams of test patterns having a single column,row, or diagonal of a first color surrounded by a second color.

FIGS. 17 and 18 are diagrams of test patterns having alternating columnsor rows of two colors.

FIG. 19 is a diagram of a checkerboard test pattern.

FIGS. 20 and 21 are diagrams of test patterns with three-columnsequences and three-row sequences, respectively.

FIG. 22 is a flow chart of a method for estimating color crosstalkcoefficients.

FIG. 23 is a flow chart of a method for adjusting a color image based onthe color crosstalk coefficients.

FIG. 24 is a flow chart of a method of collecting data from colorcrosstalk test patterns to estimate color correction coefficientsincluded in a camera for in-camera crosstalk correction.

FIG. 25 is a diagram showing color crosstalk in a portion of a Bayerpattern color image sensor array.

FIGS. 26A to 26 f show alternative test patterns that may be used insome embodiments.

FIGS. 27A and 27B show an example of test data from collected from greenand red monochrome light sources, respectively.

DETAILED DESCRIPTION

This description of the exemplary embodiments is intended to be read inconnection with the accompanying drawings, which are to be consideredpart of the entire written description. In the description, relativeterms such as “lower,” “upper,” “horizontal,” “vertical,” “above,”“below,” “up,” “down,” “top” and “bottom” as well as derivative thereof(e.g., “horizontally,” “downwardly,” “upwardly,” etc.) should beconstrued to refer to the orientation as then described or as shown inthe drawing under discussion. These relative terms are for convenienceof description and do not require that the apparatus be constructed oroperated in a particular orientation.

In some embodiments, a method and apparatus correct the pixel-to-pixelcrosstalk in color image sensors. Referring to FIGS. 1A and 1B, a suiteof color-filter-based test patterns 120 are embedded on-chip in the sameintegrated circuit 100 or 100′ that includes the color image sensor 110.The color-filter-based test patterns 120 are proximate to the colorimage sensor, also referred to herein as the primary pixel array. Thepixels of the test patterns have identical structures to the pixels ofthe primary array in every aspect, such as the active transistor layoutand the interconnect metal wires, except the color filters. A set ofparameters characterizing the pixel-to-pixel crosstalk are extractedfrom the measurement results on the test patterns 120. The set ofcrosstalk parameters are used in crosstalk correction as a part of theimage signal processing (ISP) procedures performed either on-chip oroff-chip, by either hardware or software.

FIGS. 1A and 1B show two examples of an integrated circuit 100, 100′,comprising: a semiconductor substrate 102, a color image sensor array110 on the substrate, and at least one color crosstalk test pattern 120on the substrate 102 proximate the color image sensor array.

The substrate 102 may be a silicon substrate, or other semiconductorsubstrate suitable for forming a color image sensor. In someembodiments, the substrate is transparent, to permit back-sideillumination of the sensors. In some embodiments, the substrate isneither completely transparent nor completely opaque. Part of the lightis absorbed and part of it passes through, depending on the absorptioncoefficient and the thickness of the substrate. Some embodiments havefront-side illuminated (FSI) sensors, and others have back-sideilluminated (BSI) sensors. For FSI sensors, part of the incoming lightis blocked by front-side metal wires and the transistors within eachpixel. The so-called fill-factor is much less than 100%. For BSIsensors, the fill-factor can be very close to 100%. The wafer is thinnedfrom several hundreds of microns to a few microns such that the lightcan reach the photodiode p-n junction before absorbed and attenuated inthe thick substrate.

The color image sensor array 110 has a first configuration of colorpixels for collecting color image data. The photosensitive elements inthe array may be complementary metal oxide semiconductor (CMOS) devicesor charge coupled devices (CCD). Each pixel has a respective colorfilter over it, acting as a band pass filter to permit light of aparticular color pass through. The color pixels of the color imagesensor array 110 are arranged in a first configuration. In someembodiments, the color filters of the color image sensor array 110 arearranged in a Bayer pattern which alternates between rows of red-greenrows and green-blue rows, so that each square of four pixels has onefiltered red, one blue, and two green pixels. A Bayer pattern isadvantageous in color image sensors, because the human eye is moresensitive to green light than to red or blue. However, other embodimentsuse different patterns. For example, instead of red, green and bluefilters, the color image sensor array 110 may have cyan, magenta andyellow filters. Also, for any given color space (e.g., RGB) manycolor-filter patterns are used in the industry; some examples are shownin FIG. 26A-F. The method described herein may use any of thesepatterns.

The color image sensor array 110 may have any of a variety of sizes,typically determined by the type of imaging device into which the IC100, 100′ is to be installed. For example, a low resolution camerasuitable for a cell phone or web cam may have a resolution of less thanone megapixel, but a professional digital camera may have a resolutionof 20 megapixels or more.

At least one crosstalk test pattern 120 is located on the substrate 102proximate the color image sensor array 110. The crosstalk test pattern120 includes a plurality of color sensing pixels arranged for makingcolor crosstalk measurements. The test pattern configuration isdifferent from the first configuration. In some embodiments, a pluralityof respectively different crosstalk test patterns 120 are provided onthe substrate 102, each of the plurality of crosstalk test patterns 120having a configuration that is different from the first configuration(of the color image sensor array 110). FIGS. 8 to 21 show non-limitingexamples of test patterns that may be used alone or in any combination.FIG. 7 shows an additional monochrome test pattern that may be used as acontrol, for collecting light of a single frequency, so that crosstalkcomponents in each pixel cancel each other out.

FIGS. 1A and 1B show two optional locations for the crosstalk testpatterns 120, but these locations are not limiting. For example, thetest patterns 120 may be located on any one, two, three or all foursides of the color image sensor array 110. The test patterns 120 arepositioned proximal to the color image sensor array, so that the testpatterns 120 are subjected to the same process conditions as the colorimage sensor array 110. To the extent that there may be any within-waferprocess variations causing non-uniform material deposition, removal ortreatment across the wafer, placement of the test patterns 120 proximatethe color image sensor array 110 minimizes these variations between agiven array 110 and its corresponding test patterns 120. Thus, each setof test patterns 120 is subjected to substantially the same process asits respective color image sensor array 110.

By providing individual sets of color crosstalk test patterns 120 foreach respective color image sensor array 110, greater color fidelity canbe obtained by using proper crosstalk-correction algorithms, regardlessof the presence or absence of process variations that may causewithin-wafer non-uniformities and/or between wafer non-uniformities.

FIG. 2 is a block diagram of an example of a system for estimating colorcrosstalk coefficients for correction of data from an individual colorimage sensor array 110. The system includes the IC 100 (or 100′) of FIG.1, or another IC having a color image sensor array 110 and anotherarrangement of color crosstalk test patterns 120 proximate to the array.The system further comprises a circuit 150 for estimating colorcrosstalk correction coefficients based on output signals from the atleast one crosstalk test pattern.

In some embodiments, the color crosstalk coefficient estimation circuitis embodied in one or more modules of a computer program executed on acomputer 140 external to the IC 100.

For example, the crosstalk among nearest neighboring pixels can bedescribed by the following first-order model. FIG. 25 shows an exampleof a blue pixel (B) surrounded by red (R) and green pixels. The greenpixels are designated Gr if located in rows of alternating red and greenfilters, and are designated Gb if located in rows of alternating blueand green filters. The blue pixel (B) in FIG. 25 receives additionallight due to eight crosstalk components, indicated by incoming arrows,and also causes crosstalk components received by the eight surroundingpixels, shown by the outgoing arrows. In some sensors, there may also besecond order crosstalk (i.e., light sensed at pixel MN due to lightimpinging on pixel MN−2 or MN+2). As long as test results do not showthese second order crosstalk components to substantially affect thecolor correction, they can be ignored. One of ordinary skill can comparethe test results with any given model to determine whether the modelprovides a desired level of accuracy. In a case where the second ordercrosstalk appears to be substantial, the model presented below canreadily be modified to further include the second order affects.

In the first order model, R_(2m,2n) stands for the ideal R pixel atcoordinates (2 m, 2 n); R*_(2m,2n) stands for the real R pixel atcoordinates (2 m, 2 n) mixed with neighboring pixels by crosstalk.

This model is described by 3 crosstalk parameters (a_(x), a_(y), d).R* _(2m,2n)=(1−a _(x) −a _(y) −d)R _(2m,2n)+(a _(x)/2)G _(2m−1,2n)+(a_(x)/2)G _(2m+1,2n)+(a _(y)/2)G _(2m,2n−1)+(a _(y)/2)G _(2m,2n+1)+(d/4)B_(2m−1,2n−1)+(d/4)B _(2m+1,2n−1)+(d/4)B _(2m−1,2n+1)+(d/4)B _(2m+1,2n+1)G* _(2m+1,2n)=(1−a _(x) −a _(y) −d)G _(2m+1,2n)+(a _(x)/2)R _(2m,2n)+(a_(x)/2)R _(2m+2,2n)+(a _(y)/2)B _(2m+1,2n+1)+(a _(y)/2)B_(2m+1,2n+1)+(d/4)G _(2m,2n−1)+(d/4)G _(2m,2n+1)+(d/4)G_(2m+2,2n−1)+(d/4)G _(2m+2,2n+1)*G _(2m,2n+1)=(1−a _(x) −a _(y) −d)G _(2m,2n+1)+(a _(x)/2)B_(2m−1,2n−1)+(a _(x)/2)B _(2m+1,2n−1)+(a _(y)/2)R _(2m,2n)+(a _(y)/2)R_(2m,2n+2)+(d/4)G _(2m,1,2n)+(d/4)G _(2m−1,1,2n+2)+(d/4)G_(2m−1,2n+2)+(d/4)G _(2m+1,2n+2)B* _(2m+1,2n+1)=(1−a _(x) −a _(y) −d)B _(2m+1,2n+1)+(a _(x)/2)G_(2m,2n+1)+(a _(x)/2)G _(2m+2,2n+1)+(a _(y)/2)G _(2m+1,2n)+(a _(y)/2)G_(2m+1,2n)+(a _(y)/2)G _(2m+1,2n+2)+(d/4)R _(2m,2n)+(d/4)R_(2m,2,2n)+(d/4)R _(2m,2n+2)+(d/4)R _(2m+2,2n+2)

Once the crosstalk parameters are known, the original R, G, B data canbe recovered, or corrected, from the crosstalk-mixed R*, G*, B* datathrough a mathematical process.

For example, the 2-dimensional space-domain linear transformation can beFourier-transformed into spatial-frequency domain. The inverse of thematrix is calculated in the spatial-frequency domain and thentransformed back into the space domain by inverse Fourier transform.

The final inverse transform may be approximated to involve only thenearest neighboring pixels to simplify the hardware or softwarecalculation.

In some embodiments, a suite of color-filter based test patterns areused to extract the crosstalk parameters experimentally. A non-limitingexample of such a suite is described below in the discussion of FIGS.7-21. For example, the parameters are obtained by best-fitting themeasured data from the test patterns 120 with the results predicted bythe mathematical model. The parameters are to be used for crosstalkcorrection in the new image processing pipe.

In other embodiments, the color crosstalk coefficient estimation may beperformed by an application specific integrated circuit (not shown).

Referring again to FIG. 2, during testing (e.g., wafer acceptancetesting, or individual IC testing), a plurality of colored lights areapplied to the color crosstalk test pattern arrays 120 from coloredlight sources 160 (which may be red, green and blue lasers emittinglight perpendicular to the test pattern arrays 120). The color crosstalkcorrection coefficients are calculated in the circuit 150, as discussedabove. A color crosstalk correction coefficient storage device 130associated with the IC 100 is provided for storing the outputs of thecolor crosstalk correction coefficient estimation circuit 150. In someembodiments, the color crosstalk correction coefficient storage deviceassociated with the IC 100 may comprise any non-volatile storage deviceaccessible by the processor 140, including for example, a hard diskdrive or flash memory drive. For example, in an embodiment in which thecolor image sensor is installed in a computer, it may be convenient tostore the color crosstalk coefficients in a memory device in thecomputer. If the color crosstalk correction coefficients are not storedin the IC 100, then they can optionally be temporarily stored in anintermediate storage device (not shown), to be copied subsequently intoa storage device in the camera, into which IC 100 is to be installed.For example, if test data are collected during wafer level testing, thesignals collected from the color correction test patterns 120 may bestored in a storage device associated with the test equipment, andsubsequently, the coefficients for each individual IC 100 can be storedin the respective color crosstalk correction coefficient storage device130 associated with each individual IC 100.

FIG. 3 is a block diagram of a digital camera 180, in which the IC 100containing the color image sensor array 110, and the color crosstalkcoefficient storage device 130 are installed. FIG. 3 shows the imagesignal processing (ISP) outside of the sensor. In the configuration ofFIG. 3, the sensor 100 without an integrated ISP unit may be referred toas a “raw-data” sensor. The camera may be a standalone still image orvideo camera, or a multi-function device with camera functions, such ascellular phones, smart phones, personal digital assistants, web cams,laptop computers, or the like. The color image data is provided by thecolor image sensor array 110 to the color image processing circuit 170.The color image processing circuit 170 may be an application specificdigital signal processor or a specially programmed general purposeprocessor.

In some embodiments, as shown in FIG. 3A, the image signal processing(ISP) unit 170 is a part of the image sensor IC 100″. Sensor 100″ withan integrated ISP unit 170′ may be referred to as a system-on-chip (SoC)sensor. Thus, FIG. 3A shows the SoC sensor 100″ with integrated ISP170′. In some embodiments (as shown in FIG. 3B, the camera 180″′includes a sensor 100″ with integrated ISP unit 170 and a separateprocessor 140′ that performs the color crosstalk coefficient estimation.

In either case, the color image processing circuit 170, 170′ uses thecolor crosstalk coefficients to perform color crosstalk corrections.Details of the image processing are described below with reference toFIG. 6.

In other embodiments (e.g., FIGS. 4 and 5) the color crosstalkcorrection coefficients are stored in non-volatile memory elements(e.g., registers) 130′ on the same substrate as the color image sensorarray 110. The on-chip storage device may include registers, flashmemory, or the like. As shown in FIG. 4, during testing, the colorcrosstalk test patterns 120 are exposed to light at various frequenciesfrom colored light source(s) 160. The signals output by the testpatterns 120 are exported to a color crosstalk coefficient estimationcircuit 150, which may be in an external computer 140. Once the colorcrosstalk correction coefficients are calculated, they are returned tothe on-chip storage device 130′.

FIG. 4 shows an external processor 140 configured to perform colorcrosstalk coefficient estimation. The color crosstalk coefficientestimation of FIG. 4 may be implemented in a variety of ways (1) In someembodiments, color crosstalk coefficient estimation is performed by aseparate, computerized testing system (e.g., before the camera or sensorchip is shipped out of the factory) as shown in FIG. 2, the coefficientsbeing stored on-chip, or off-chip but inside the camera system. (2) Insome embodiments, color crosstalk coefficient estimation is performed bysoftware running on a CPU inside the camera 180, which may be on anotherIC separate from the sensor chip, as shown in FIG. 3B. (3) In someembodiments, color crosstalk coefficient estimation is performed byhard-wired digital logic circuits on-chip of the sensor (e.g., if theSoC sensor 100″ of FIG. 3A is used).

FIG. 5 shows a camera 180′ in which the IC 100′ containing the colorimage sensor array 110, the test patterns 120 and the color crosstalkcorrection coefficient storage device 130′ are installed. The digitalcamera 180′ comprises a housing 182 containing the integrated circuit100. A storage device 130′ within the housing 182 stores a plurality ofcolor crosstalk correction coefficients that are based on output signalsfrom the at least one crosstalk test pattern. A color image processingcircuit 170 including a crosstalk correction circuit 202 (FIG. 6) isprovided within the housing 182 for receiving color image data from thecolor image sensor array 110. The color image processing circuit 170uses the color crosstalk coefficients to perform color crosstalkcorrections.

FIG. 6 is a block diagram of the image processing system 170 in thecameras of FIGS. 3 and 5. The inputs to the image processor 170 includethe image data from the color image sensor array 110 and the colorcrosstalk correction coefficients from the color correction coefficientstorage device 130.

Block 200 includes image pre-processing. This may include, for example,correction of column fixed pattern noise (FPN) or pixel level FPN,correction for defective pixels, dark current correction, or the like.Additional examples of pre-processing may include: linearization tocorrect the non-linearity of the raw sensor response; de-noising circuitto reduce temporal noises; lens-shading correction to correct thenon-uniform effects caused by global lens, or any combination of thesepre-processing operations.

Block 202 includes applying the color crosstalk correction coefficientsfrom the color correction coefficient storage device 130 to correct thesignal levels of the various pixels to remove the crosstalk effects. Insome embodiments, this color crosstalk correction step is appliedseparately from color correction. For example, the crosstalk correctionon one pixel uses the data from neighboring pixels, as discussed herein.Since the video stream comes in continuously, line buffers (SRAM) savethe data of a few previous lines for this purpose. In contrast, colorcorrection may be performed using the RGB data of one pixel. The abilityto separate color crosstalk correction from color correction permits useof different processing resources and algorithms to improve performanceand/or accuracy of the circuits that perform each operation.

Block 204 performs color interpolation. Because each pixel only samplesone color (e.g., R, G, o B), color interpolation calculates theremaining two color values at each pixel. For example, a red pixel has ameasured red value, but the blue and green levels at each red pixel areobtained by color interpolation. In some embodiments, the interpolationalgorithms are adaptive. In some embodiments, the particular choice ofcolor interpolation algorithm may depend on the image.

Block 206 performs white balance to correct the red and bluecoefficients that combine to generate white. White balance may beperformed using a an automatic white balance function that searches fora gray object in the image, by a camera pre-set, or on an image by imagebasis using custom white balance data collected by the photographer(e.g., by taking a picture of a white object or gray card) at the timethe image was collected.

Block 208 performs color correction or enhancement. Color correctionaffects the image utilizing control over intensities of red, green,blue, gamma (mid tones), shadows (blacks) and highlights (whites).Additional corrections may be used to change luminance, saturation andhue in six colors (red, green, blue, cyan, magenta, yellow). In someembodiments, special digital filters and effects may also be applied tothe images. Because the color crosstalk correction is performed by block202 before the color correction block 208, the color correction blockcan be focused and optimized for giving the image a desired style ormood.

Block 210 performs post-processing. A variety of post-processingfunctions may be included for in-camera or in-computer post-processing,such as contrast adjustments, cropping or the like. Other examples ofpost-processing include, but are not limited to: edge enhancement, Gammacorrection for display, subsampling for preview, color space conversion,for example, converting {R, G, B} to {Y, Cb, Cr} for JPEG, MPEG,motion-JPEG compression; 444-to-422 down sampling.

FIGS. 7-21 show a variety of test patterns which may be used. In FIGS.7-21, individual pixels are indicated by squares having the letters R,G, or B. For brevity, the drawings do not show every possiblepermutation of colors. Rather, an exemplary pattern is shown. Theletters R, G and B are considered to represent first, second and thirdcolors in a color space, such as RGB or CMY. In a single-color pattern(FIG. 7), the R pixels may be replaced with G or B pixels. For thetwo-color patterns of FIGS. 8-19, R-G may be replaced with G-R, R-B,B-R, B-G, G-B, C-M, M-C, C-Y, Y-C, M-Y or Y-M. R-B may be replaced withB-R, R-G, G-R, B-G, G-B, C-M, M-C, C-Y, Y-C, M-Y or Y-M; and G-B may bereplaced with G-R, R-B, B-R, R-G, G-R, C-M, M-C, C-Y, Y-C, M-Y or Y-M.Similarly, in the three-color patterns of FIGS. 20 and 21, RGB can bereplaced with CMY. In other embodiments, a “W” (white, or clear) pixelmay also be used in the color filter array, as discussed below withreference to FIGS. 26A-26F.

Also, the examples of FIGS. 7-21 show 8×8 test patterns. The testpatterns can be other sizes. Larger test patterns permit experimentationwith higher order color crosstalk effects. Smaller patterns use lessspace and allow smaller overall chip footprint.

Whereas FIGS. 1-5 refer to test patterns 120 generally, FIGS. 7-21 referto specific examples of test patterns 70, 72, 74, 76, 78, 80, 82, 84,86, 88, 90, 92, 94, 96, 98. These specific test patterns may besubstituted in the locations where FIGS. 1-5 show test patterns 120.

FIG. 7 shows a test pattern 70 of pixels of uniform color proximate thecolor image sensor array for measuring pixel color response in theabsence of color crosstalk. For example, the pixels in the center regionof the mini array are surrounded by pixels of the same colors;therefore, the crosstalk among neighboring pixels cancel with each otherunder uniform illumination. These can be called reference color pixels.Although red pixels are shown, a complete set of monochrome patterns forthe RGB color space further includes a pattern of green pixels and apattern of blue pixels. Alternatively, in a CMY color space embodiment,three monochrome test patterns of cyan, magenta and yellow pixels areprovided. Alternatively, other color spaces (such as WRGB, KodakTruesense pattern, etc.) may be used, such as those in FIGS. 26A-26F.

FIGS. 8-21 show a plurality of color crosstalk test patterns, each testpattern including: one or more pixels of a first color in a firstarrangement; and a plurality of neighboring pixels around the firstarrangement, where the neighboring pixels are different in color fromthe first color.

FIG. 8 shows a test pattern 72 in which a single pixel of a first coloris surrounded by pixels of a second color (different from the firstcolor) filling the remainder of the test pattern. FIG. 8 shows a singlegreen pixel within a field of blue pixels. Similar patterns are providedfor a blue pixel surrounded by green pixels, a green pixel surrounded byred, a red pixel surrounded by green, a red pixel surrounded by blue anda blue pixel surrounded by red.

FIGS. 9-12 show four patters in which a first color of pixel fills oneside of the test pattern, and a second color fills the other side of thetest pattern. These figures include a horizontal interface (FIG. 9)between rectangular regions of different color; a vertical interface(FIG. 10) between rectangular regions of different color; and a diagonalinterface between substantially triangular or trapezoidal regions ofdifferent color (FIGS. 11 and 12).

In FIG. 9, the left half of the test pattern 74 is occupied by redpixels, and the right half is occupied by green pixels.

In FIG. 10, the bottom half of the test pattern 76 is occupied by redpixels, and the top half is occupied by green pixels.

In FIG. 11, the bottom-left half of the test pattern 78 is occupied byred pixels, and the top-right half is occupied by green pixels.

In FIG. 12, the top-left half of the test pattern 80 is occupied by redpixels, and the bottom-right half is occupied by green pixels.

As noted above, the use of red and green in FIGS. 9-12 is exemplary, andeach of the four patterns is representative of six differentcombinations of two colors from the group consisting of red, green andblue. In other embodiments, each of FIGS. 9-12 represents six differentcombinations of cyan, magenta and yellow pixels. Thus, FIGS. 9-12represent 24 different test patterns.

FIGS. 13-16 show exemplary patterns 82, 84, 86 and 88, wherein theplurality of unique crosstalk test patterns include a pattern having asingle row (FIG. 13), column (FIG. 14) or diagonal line of pixels (FIGS.15, 16) of the first color surrounded by pixels of a second colordifferent from the first color.

In FIG. 13, the test pattern 82 includes a single column of red pixelsin a test pattern where the remaining pixels are all blue.

In FIG. 14, the test pattern 84 includes a single row of red pixels in atest pattern where the remaining pixels are all blue.

In FIG. 15, the test pattern 86 includes a single diagonal line of redpixels (rising from left to right) in a test pattern where the remainingpixels are all blue.

In FIG. 16, the test pattern 86 includes a single diagonal line of redpixels (falling from left to right) in a test pattern where theremaining pixels are all blue.

FIGS. 17 and 18 show examples of test patterns 90 and 92, wherein theplurality of unique crosstalk test patterns include a pattern having oneof the group consisting of: rows of pixels (FIG. 17) alternating betweenthe first color and a second color different from the first color; andcolumns of pixels (FIG. 18) alternating between the first color and thesecond color.

In FIG. 17, the pattern 90 includes alternating columns of red andgreen.

In FIG. 18, the pattern 92 includes alternating rows of red and green.

FIG. 19 shows an example of a pattern 94, wherein the plurality ofunique crosstalk test patterns include a checkerboard pattern havingfirst and second colors different from each other. In FIG. 19, thecheckerboard comprises alternating red and green pixels.

The use of red and green in FIGS. 13-19 is exemplary, and each of theseven patterns is representative of six different combinations of twocolors from the group consisting of red, green and blue. In otherembodiments, each of FIGS. 13-19 represents six different combinationsof cyan, magenta and yellow pixels.

FIGS. 20 and 21 are examples of patterns 96 and 98, wherein theplurality of unique crosstalk test patterns include a pattern having oneof the group consisting of: a plurality of sets of pixel rows (FIG. 20),each set including a first row of pixels of the first color, a secondrow of pixels of a second color different from the first color, and athird row of pixels of a third color different from the first and secondcolors; or a plurality of sets of pixel columns (FIG. 21), each setincluding a first column of pixels of the first color, a second columnof pixels of the second color, and a third column of pixels of the thirdcolor. Although red, blue and green rows or columns are shown, in otherembodiments, the rows (or columns) include cyan, magenta and yellowpixels, respectively.

FIGS. 26A-26F show alternative patterns which may be included in thetest pattern suite. FIG. 26A shows a “KODAK TRUESENSE” pattern,including R, G, B and W pixels. FIG. 26B shows a variation of thepattern in FIG. 26A. FIG. 26C shows a “SONY CLEARVID” pattern, whichuses R, G and B pixels in a different arrangement from the Bayerpattern, with a ratio of 6 green pixels to each red and blue pixel. FIG.26D shows a TOSHIBA WRGB pattern, which has a different arrangement ofR, G, B and W pixels, including equal numbers of R, G, B and W pixels.FIGS. 26E and 26F show alternative test patterns, including two greenpixels for each red and blue pixel, and two white pixels for each greenpixel. In various embodiments, the suite of test patterns may includeany combination of one or more of these patterns.

A set of 74 test patterns is described above, including three ofpatterns 70, six each of patterns 72, 74, 76, 78, 80, 82, 84, 86, 88,90, and 92, three of pattern 94, and one of each pattern 96 and 98.Individual test configurations may include a subset of these 74 testpatterns, if satisfactory color crosstalk correction is achieved.

FIGS. 27A and 27B schematically show collection of test data. In FIG.27A, a green light source 2702 exposes the test pattern 74 to greenmonochrome light. The test data 2704 show the light values 2706 measuredwithout crosstalk (e.g., using pattern 70, FIG. 7) and the smooth curve2708 collected from pattern 74, with crosstalk. Similarly, in FIG. 27B,a green light source 2712 exposes the test pattern 74 to red monochromelight. The test data 2714 show the light values 2716 measured withoutcrosstalk (e.g., using pattern 70, FIG. 7) and the smooth curve 2718collected from pattern 74, with crosstalk. Although FIGS. 27A and 27Bshow the light radiating in many directions (which would increase colorcrosstalk), the light may be all perpendicular to the test patterns 70,74, to minimize the color crosstalk during testing. Alternatively, bothperpendicular and non-perpendicular light may be used.

FIG. 22 is a flow chart of a method for estimating color coefficients.

At step 2200, color crosstalk test pattern data are collected from atleast one test pattern 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92,94, 96, 98 of color sensing pixels on an integrated circuit (IC) 100having a color image sensor array 110 thereon proximate the at least onetest pattern. In some embodiments, the collecting includes collectingcolor crosstalk test pattern data from a plurality of respectivelydifferent color crosstalk test patterns on the same IC as the colorimage sensor array.

At step 2202, color crosstalk correction coefficients are estimated,based on the crosstalk test pattern data, for correcting spatialspectral crosstalk in color image data to be collected by the colorimage sensor array. In some embodiments, the color crosstalk testpattern data are exported to a processor 140 external to the camera 180,and the estimated coefficients are programmed into the storage device bythe external processor.

In some embodiments, the estimation is performed using the algorithmdescribed above. The above-described coefficient estimation algorithm isonly one example. In other embodiments, different models of greater orlesser complexity are used.

For example, in one embodiment, the data (R, Gr, Gb, B) are measured ona test pattern set as described below with reference to FIG. 7, wherethe crosstalk among neighboring pixels cancel out with each and the datamatch those under an ideal condition without any crosstalk.

The data (R*, Gr*, Gb*, B*) are measured on the Bayer pattern of theprimary pixel array where crosstalk takes place.

The two sets of data are taken under several different light sources,for instance, red, green, and blue light using narrow-band or broad-bandfilters, or a sequence of narrow-band monochromatic lights sweeping theentire visible spectrum from 400 nm to 740 nm. The crosstalk parameters(ax, ay, d) are to be extracted by least root-mean-square (RMS) fittingof the 2 sets of data, with and without crosstalk.

In some embodiments, the light source(s) 160 is positioned to providecollimated perpendicular light to the test patterns to reduce opticalcrosstalk during the testing. In other embodiments, the light angle ofincidence is varied, to provide additional data regarding the impact ofnon-perpendicular light on color crosstalk.

If the difference between Gr and Gb is small and can be ignored, thatis, ax=ay, the equation can be further simplified to below for parameterextraction.

$\begin{pmatrix}R^{*} \\G^{*} \\B^{*}\end{pmatrix} = {\begin{pmatrix}{1 - {2a} - d} & {2a} & d \\a & {1 - {2a}} & a \\d & {2a} & {1 - {2a} - d}\end{pmatrix}\begin{pmatrix}R \\G \\B\end{pmatrix}}$

These coefficients are stored in a storage device 130 associated withthe digital camera 180 in which the color image sensor array and thestorage device are to be installed. In some embodiments, thecoefficients are stored in a storage device on the same IC 100 as thecolor image sensor array 110. In other embodiments, the coefficients arestored in a storage device associated with an external processor.

A color crosstalk correction circuit 202 is coupled to receive theestimated coefficients and the color image data from the color imagesensor array 110. The color crosstalk correction circuit 202 isconfigured to adjust the color image data based on the estimatedcoefficients.

FIG. 23 is a flow chart of a method of performing color crosstalkcorrection.

At step 2300, estimated color crosstalk coefficients associated with anintegrated circuit (IC) are received, where the estimated colorcrosstalk coefficients are based on crosstalk test pattern datacollected by a test pattern of color pixels proximate a color imagesensor array on the IC.

At step 2302, a color image collected by the color image sensor array isadjusted, using the estimated color crosstalk coefficients. In someembodiments, the adjusting step is performed by a digital camera, andthe estimated color crosstalk coefficients are received from a storagedevice in the digital camera.

FIG. 24 is a flow chart of a method of using the test patterns for aprocess monitor.

At step 2400, the test patterns 120 are exposed to perpendicularmonochrome light at a plurality of different prescribed frequencies.

At step 2402, color crosstalk test pattern data are collected from theplurality of test patterns on a plurality of different IC chips on thewafer (and optionally on multiple wafers).

At step 2404, the color crosstalk coefficients are estimated for eachIC.

At step 2406, the coefficients are stored in the storage deviceassociated with the IC 100.

At step 2408, the color crosstalk coefficients from different ICs arecompared.

At step 2410, differences in color crosstalk coefficients within andbetween wafers are identified as indicators of process non-uniformity.Differences in coefficients between chips on the same wafer is anindication of a spatial non-uniformity of a recipe parameter (e.g.,temperature, pressure, bias voltage, mass flow rate, etc.) at a givenpoint in time. Differences in coefficients between wafers indicate adrift of a process parameter over time.

At step 2412, assuming the IC 100 having the color image sensor array110 and the color crosstalk test patterns 120 is a good chip, the IC isinstalled in a camera.

At 2414, a color crosstalk correction circuit is provided to receiveimage data from the color image sensor array 110 and the color crosstalkcorrection coefficients from the storage device 130. The color crosstalkcorrection circuit corrects the image data based on the coefficients, toremove the effects of color crosstalk. In some embodiments, the colorcrosstalk correction circuit is provided within the camera. In otherembodiments, the color crosstalk correction circuit is provided in anexternal computer. In some embodiments, the color crosstalk correctioncircuit is provided both within the camera and within an externalcomputer. For example, the camera may permit the user to produce JPEGimages within the camera, including a first color crosstalk correction,and/or produce RAW image files that are processed in an externalcomputer, where the external computer applies a second color crosstalkcorrection that can be the same as or different from the first colorcrosstalk correction.

A variety of embodiments are described above. In some embodiments, anintegrated circuit 100 comprises a semiconductor substrate 102 and acolor image sensor array 110 on the substrate. The color image sensorarray 110 has a first configuration of color pixels for collecting colorimage data. At least one crosstalk test pattern 120 is provided on thesubstrate 102 proximate the color image sensor array 110. The crosstalktest pattern 120 includes a plurality of color sensing pixels arrangedfor making color crosstalk measurements. The test pattern configurationis different from the first configuration.

In some embodiments, a method comprises: collecting color crosstalk testpattern data from at least one test pattern of color sensing pixels onan integrated circuit (IC) having a color image sensor array thereonproximate the at least one test pattern; and estimating color crosstalkcoefficients based on the crosstalk test pattern data, for correctingspatial spectral crosstalk in color image data to be collected by thecolor image sensor array.

In some embodiments, a method comprises: (a) receiving estimated colorcrosstalk coefficients associated with an integrated circuit (IC), wherethe estimated color crosstalk coefficients are based on crosstalk testpattern data collected by a test pattern of color pixels proximate acolor image sensor array on the IC; and (b) adjusting a color imagecollected by the color image sensor array, using the estimated colorcrosstalk coefficients.

Although the subject matter has been described in terms of exemplaryembodiments, it is not limited thereto. Rather, the appended claimsshould be construed broadly, to include other variants and embodiments,which may be made by those skilled in the art.

What is claimed is:
 1. A digital camera comprising: a color image sensorarray capable of collecting a color image, the color image sensor arrayhaving a first configuration of color pixels for collecting color imagedata; one or more respectively different color crosstalk test patterns,each color crosstalk test pattern having a respective configuration thatis different from the first configuration for providing signals used togenerate color crosstalk correction coefficients; and a color crosstalkcorrection circuit for adjusting the color image collected by the colorimage sensor array, using the color crosstalk correction coefficients.2. The digital camera of claim 1, wherein the color image sensor arrayand the one or more color crosstalk test patterns are proximate to eachother on an integrated circuit substrate.
 3. The digital camera of claim2, wherein the integrated circuit includes a storage device for storingthe color crosstalk correction coefficients.
 4. The digital camera ofclaim 3, wherein the integrated circuit includes the color crosstalkcorrection circuit for adjusting the color image.
 5. The digital cameraof claim 3, wherein the digital camera further comprises: a housingcontaining the integrated circuit; and a color image processing circuitwithin the housing, the color image processing circuit including thecolor crosstalk correction circuit for adjusting the color image.
 6. Thedigital camera of claim 2, wherein the color crosstalk test patterns arearranged adjacent at least one edge of the color image sensor array. 7.The digital camera of claim 1, wherein the one or more color crosstalktest patterns includes a plurality of unique crosstalk test patterns,each color crosstalk test pattern including: one or more pixels of afirst color in a first arrangement; a plurality of neighboring pixelsaround the first arrangement, the neighboring pixels being different incolor from the first color.
 8. The digital camera of claim 1, furthercomprising a circuit for estimating the color crosstalk correctioncoefficients based on output signals from the at least one crosstalktest pattern.
 9. The digital camera of claim 1, wherein the digitalcamera comprises an integrated circuit that contains the color imagesensor array and the storage device containing color crosstalkcorrection coefficients.
 10. The digital camera of claim 1, furthercomprising a storage device containing color crosstalk correctioncoefficients, wherein the digital camera includes an integrated circuitcontaining the color image sensor, the storage device and the colorcrosstalk correction circuit, and the camera is configured to receivethe color crosstalk correction coefficients from a processor external tothe integrated circuit, and store the received color crosstalkcorrection coefficient data in the storage device.
 11. The digitalcamera of claim 1, further comprising a color image processor containingthe color crosstalk correction circuit for adjusting the color image,the color image processor further comprising: an image pre-processingmodule configured to receive image data from a plurality of imagingelements in a color image sensor and output pre-processed data to thecolor crosstalk correction circuit; a color interpolation module coupledto receive data from the color crosstalk correction module; a whitebalance correction module coupled to the color interpolation module; anda color correction module coupled to the white balance correctionmodule.
 12. The digital camera of claim 11, wherein, the pre-processingmodule is configured to perform at least one of the group consisting offixed pattern noise correction, correction of defective pixels, or darkcurrent correction, before providing the image data to the colorcrosstalk correction circuit.
 13. A color image processor, comprising:an image pre-processing module configured to receive image data from aplurality of imaging elements in a color image sensor and outputpre-processed data, the color image sensor array having a firstconfiguration of color sensors for collecting color image data; and acolor crosstalk correction circuit for correcting color crosstalkbetween neighboring imaging elements of different colors from each otherin the pre-processed data, using color crosstalk correction coefficientsbased on signals output by one or more respectively different colorcrosstalk test patterns having respective configurations that aredifferent from the first configuration.
 14. The color image processor ofclaim 13, wherein the color crosstalk correction circuit is configuredto receive the color crosstalk correction coefficients from a storagedevice associated with the color image sensor, and to apply the colorcrosstalk correction coefficients when correcting the color crosstalk.15. The color image processor of claim 13, wherein the color crosstalkcorrection circuit is configured to apply a first order color crosstalkcorrection algorithm to the image data.
 16. The color image processor ofclaim 13, further comprising: a color interpolation module coupled toreceive data from the color crosstalk correction circuit; a whitebalance correction module coupled to the color interpolation module; anda color correction module coupled to the white balance correctionmodule.
 17. The color image processor of claim 13, further comprising acircuit for estimating color crosstalk correction coefficients based onoutput signals from the at least one crosstalk test pattern.
 18. Anintegrated circuit, comprising: a color image sensor array having afirst configuration of color sensors for collecting color image data;and one or more respectively different color crosstalk test patternsproximate the color image sensor array and having respectiveconfigurations that are different from the first configuration.
 19. Theintegrated circuit of claim 18, further comprising a storage deviceencoded with color crosstalk correction coefficients to be applied forcorrecting color crosstalk in image data collected by the color imagesensor array.
 20. The integrated circuit of claim 19, further comprisinga color image processor for processing the color image data, the colorimage processor configured to apply the crosstalk correctioncoefficients for correcting the color crosstalk in the image datacollected by the color image sensor array.